Selection of a preferred image from multiple captured images

ABSTRACT

Systems, methods, and computer-readable media are disclosed for selection of a preferred image from multiple captured images. An image corresponding to a photograph time t=0 may be retrieved from a circular buffer and stored as a preferred image. Alternative images captures before and after the t=0 image may be retrieves and stored in an alternative image location. The t=0 image and preferred images may be presented to a user in a user interface. The user may select a preferred image for the photograph from among the t=0 image and the alternative images.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of U.S. application Ser. No.15/137,769, filed Apr. 25, 2016, which is a continuation of U.S.application Ser. No. 14/480,308, filed Sep. 8, 2014, now U.S. Pat. No.9,325,876, issued Apr. 26, 2016, the contents of which are incorporatedherein by reference in their entireties.

BACKGROUND

Many electronic devices include cameras that enable a user to capturestill images and video of their environment. A user's perception of theperformance of a camera may be affected by multiple factors, such as thesize of an image sensor, the shutter speed, and the lighting conditions.Some cameras may be affected by the lighting conditions of theenvironment. In some instances, a photograph taken by the camera may beunsatisfactory to a user. Consequently, a user may spend additional timetaking multiple photographs in an attempt to obtain the best possibleimage. Moreover, a user may not have time to take additional photographsand may be unhappy with or discard an unsatisfactory photograph.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a system in accordance with anembodiment of the disclosure;

FIG. 2 is a schematic diagram of a user device and a portion of acircular buffer in accordance with an embodiment of the disclosure;

FIG. 3 is a schematic diagram depicting retrieval of images from aportion of the circular buffer in accordance with an embodiment of thedisclosure;

FIG. 4 is a schematic diagram depicting retrieval of images from aportion of the circular buffer in accordance with an embodiment of thedisclosure;

FIG. 5 is a schematic diagram depicting retrieval of images from aportion of the circular buffer in accordance with an embodiment of thedisclosure;

FIGS. 6A, 6B, and 6C are block diagrams of a process for determining apreferred photograph in accordance with an embodiment of the disclosure;and

FIGS. 7A-7E are diagrams of screens of a user interface in accordancewith an embodiment of the disclosure.

Certain implementations will now be described more fully below withreference to the accompanying drawings in which various implementationsand/or aspects are shown. However, various aspects may be implemented inmany different forms and should not be construed as limited to theimplementations set forth herein; rather, these implementations areprovided so that this disclosure will be thorough and complete, and willfully convey the scope of the disclosure to those skilled in the art.Like numbers in the figures refer to like elements throughout. Hence, ifa feature is used across several drawings, the number used to identifythe feature in the drawing where the feature first appeared will be usedin later drawings.

DETAILED DESCRIPTION

Embodiments of the disclosure enable a user to select among imagescaptured before, during, and after a user presses a shutter button of acamera to take a photograph. The images are displayed to the user in animage selection user interface on a user device, such as a camera or asmartphone with a camera. A user may view an image captured before theshutter button was pressed (a “pre-shutter” image), an image captured atthe time the shutter button was pressed (a “shutter” image), and animage captured after the shutter button was pressed (a “post-shutter”image). After viewing the images, the user may select a preferred image(also referred to as a “best shot” image) for the photograph. Thephotograph on the user device will be updated to the user-selected imageand may be sent to a cloud storage service for synchronization to otheruser devices.

With the foregoing in mind, described herein are systems, methods, andcomputer-readable media for selection of a preferred image from multiplecaptured images. In some embodiments, a user device may capture imagesvia a camera after a camera ready mode is activated. A sequence ofcaptured images may be captured at a frame rate and stored in a circularbuffer of a memory of the user device. A user may select a shutterbutton of the user device to take a photograph. Upon selection of theshutter button at a time t=0, the image captured at time t=0 and twoalternative images may be retrieved from the circular buffer. Forexample, the captured image stored in the circular buffer location thatcorresponds to time t=0 may be retrieved as a default preferred imagefor the photograph. The t=0 image may be stored in a preferred imagelocation. Additionally, alternative images before and after the captureof the t=0 image may be retrieved from the circular buffer.

In some embodiments, the alternative images may be selected using aframe interval determined from a frame rate of image capture of theimages stored in the circular buffer. For example, a frame rate may beobtained from the camera (e.g., from an image sensor) and a frameinterval determined from the frame rate. In some embodiments, the frameinterval may be determined to meet a specific time interval between thealternative frames and the t=0 frame. Using on the frame interval, analternative image captured before the t=0 image may be retrieved from acircular buffer location. Similarly, using the frame interval, analternative image captured after the t=0 image may be retrieved from acircular buffer location. The alternative images may be stored in analternative image location of a memory of the user device.

In some embodiments, an alternative image may be selected based on theabsence of a flash when the images where captured. In some embodiments,the t=0 image may be captured using with an activated flash of thecamera of the user device. In such embodiments, both a duration of timeand a frame interval may be used to retrieve images from the circularbuffer. In such embodiments, the alternative image before the capture ofthe t=0 image may be retrieved based on a duration before the t=0 imageto obtain an alternative image captured without the use of a flash. Thealternative image after the capture of the t=0 image may be retrievedbased on a frame interval specifying a number of frames after thecapture of the t=0 image. The activation of the flash may be receivedfrom the camera at the t=0 time the shutter button is pressed. Thealternative images may be retrieved from the circular buffer and storedin an alternative image storage location.

In some embodiments, the t=0 image and alternative images may beprovided to a user in a user interface for selection of a preferredimage for the photograph taken at t=0. In some embodiments, the t=0image may be selected as the default preferred image. Using the userinterface, a user may view the alternative images and, in someinstances, select an alternative image as the preferred image for thephotograph. The user-selected alternative image may replace the defaultt=0 image as the photograph. In some embodiments, the user-selectedalternative image may be moved to the preferred image location and theunselected 1=0 image may be moved to the alternative image location. Insome embodiments, a pointer associated with the photograph may bemodified to point to the user-selected image. In some embodiments, theuser-selected image may be transmitted to a server (e.g., one of moreservers of a cloud storage service).

FIG. 1 illustrates a system 100 that includes a user device 102 of auser in accordance with an embodiment of the disclosure. The user device102 may be a tablet computer, a smartphone, a wearable computer, apersonal digital assistant (PDA), an e-reader, or another user devicethat includes a camera. In some embodiments, the user device 102 may bea camera having the capabilities described below. The user device 102may be or include a camera to enable capture of an image 106 that mayinclude, for example, a subject 108. The image 106 may be illuminated bya light source 110 (e.g., one or more light sources, including naturaland unnatural light sources). In some embodiments, the light source 110may include a flash 111 of the user device 102. The user device 102 mayexchange information over a network 112 with one or more servers 114.The one or more servers 114 may include web servers, applicationservers, or other types of servers. In some embodiments, the one or moresevers 114 may be or may include one or more networked storage devices.Additionally, the one or more servers 114 may be, for example, serversarranged in any physical and virtual configuration, such as servers inone or more data processing centers, a distributed computingenvironment, or other configuration. Such configurations may use thenetwork 112 for communication or may communicate over other networks. Insome embodiments, the one or more servers 114 may be a part of a cloudstorage service, such as an image storage service. In such embodiments,images captured by the user device 102 may be stored on the one or moreservers 114 (e.g., images may be synchronized between the user device102 and the one or more servers 114). For example, images may betransmitted between a user device 102 and a networked storage device.While a single user device 102 is shown, in some embodiments other userdevices of a user may be present and may communicate with the one ormore servers 114. Such other user devices may also synchronize imageswith the one or more servers 114.

The image 106 may be viewed on the user device 102, and a user mayinitiate capture of the image 106 via interaction with the user device102. For example, in some embodiments a camera user interface mayinclude a user interface element, (e.g., a shutter button 116), that,when selected, initiates a photograph of the image 106. In someembodiments, a hardware button, switch, or other hardware componentacting as a shutter button may initiate a photograph of the image 106when selected by a user.

As also shown in FIG. 1, the user device 102 may also include one ormore processors 118, memory 120 (e.g., one or more memories such asvolatile memory and non-volatile memory), one or more interfaces 122, adisplay 124, and a camera 126. It should be noted that FIG. 1 depictsmerely one example of a particular implementation of a user device 102and may illustrate only some of the types of components andfunctionalities that may be present.

The processors 118 may comprise one or more cores and are configured toaccess and execute (at least in part) computer-readable instructionsstored in the one or more memories 120. The one or more processors 118may include, without limitation: a central processing unit (CPU), adigital signal processor (DSP), a reduced instruction set computer(RISC), a complex instruction set computer (CISC), a microprocessor, amicrocontroller, a field programmable gate array (FPGA), or anycombination thereof. The user device 102 may also include a chipset (notshown) for controlling communications between the one or more processors118 and one or more of the other components of the user device 102. Incertain embodiments, the user device 102 may be based on an Intel®architecture or an ARM® architecture and the processor(s) 118 andchipset may be from a family of Intel® processors and chipsets. The oneor more processors 118 may also include one or more application-specificintegrated circuits (ASICs) or application-specific standard products(ASSPs) for handling specific data processing functions or tasks.

The interfaces 122 may also comprise one or more communicationinterfaces or network interface devices to provide for the transfer ofdata between the user device 102 and another device directly such as ina peer-to-peer fashion, via an electrical network (not shown), or both.The communication interfaces may include, but are not limited to:personal area networks (“PANs”), wired local area networks (“LANs”),wireless local area networks (“WLANs”), wireless wide area networks(“WWANs”), and so forth. The communication interfaces may use anysuitable communications standard, protocol and technology, includingEthernet, Global System for Mobile Communications (GSM), Enhanced DataGSM Environment (EDGE), a 3G network (e.g., based upon the IMT-2000standard), high-speed downlink packet access (HSDPA), wideband codedivision multiple access (W-CDMA), code division multiple access (CDMA),time division multiple access (TDMA), a 4G network (e.g., IMT Advanced,Long-Term Evolution Advanced (LTE Advanced), etc.), Bluetooth, WirelessFidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g or IEEE802.11n), voice over Internet Protocol (VoIP), Wi-MAX, a protocol foremail (e.g., Internet message access protocol (IMAP) or post officeprotocol (POP)), instant messaging (e.g., extensible messaging andpresence protocol (XMPP), Session Initiation Protocol for InstantMessaging and Presence Leveraging Extensions (SIMPLE), Instant Messagingand Presence Service (IMPS)), Multimedia Messaging Service (MMS), ShortMessage Service (SMS), or any other suitable communication protocol.

The display 124 may display images in varying degrees of resolution,contrast, content, and/or location. The display 124 may be, but is notlimited to, a liquid crystal display (LCD), light emitting diode (LED)display, a lenticular display, an e-ink display, or an organic lightemitting diode (OLED) display. The display 124 may display a userinterface (e.g., a graphical user interface). In accordance with someembodiments, the display 124 may include or be provided in conjunctionwith touch sensitive elements through which a user may interact with theuser interface. Such a touch-sensitive display may be referred to as a“touch screen.”

The camera 126 may be used to capture images displayed by the userdevice 102. The camera 126 may vary in resolution, field of view, and/orshutter speed. The camera 126 may include, but is not limited to, chargecoupled device (CCD) camera, a complementary metal oxide semiconductor(CMOS) image sensor camera, an infrared camera, or an optical lenscamera.

The memory 120 (which may include one or more tangible non-transitorycomputer readable storage medium) may include volatile memory, such asrandom access memory (RAM), and non-volatile memory, such as ROM, flashmemory, a hard drive, any other suitable optical, magnetic, orsolid-state storage medium, or a combination thereof. The memory 120 maybe accessible by the one or more processors 118 and other components ofthe user device 102. The memory 120 may store a variety of informationand may be used for a variety of purposes and may store executablecomputer code. The executable computer code may include programinstructions executable by a processor (e.g., one or more processors118) to implement one or more embodiments of described herein. Thememory 120 may store an operating system 128 that includes a pluralityof computer-executable instructions that may be implemented by theprocessor 118 to perform a variety of tasks to operate the interface(s)122 and any other hardware installed on the user device 102. The memory120 may provide for storage of a camera module 130, a circular buffer132, and images (indicated by image storage 134).

The camera module 130 may provide a camera user interface for viewingand capturing images via the user device 102. The camera module 130 mayreceive or provide the capabilities of the camera 126 and determineimage parameters associated with an image. In some embodiments, imageparameters may be stored with a captured image as image metadata.Additionally, the camera module 130 may store and retrieve images fromthe circular buffer 132 in accordance with the techniques describedherein. Additionally the camera module 130 may provide a user interfacefor selection of images and may store user-selected images in the memory120.

The image storage 134 of the memory 120 may store images selected by auser for editing and viewing. In some embodiments, the image storage 134may include multiple locations for storing different images. In someembodiments, images stored in the memory 120 may be transmitted to theimage server 114 for storage in an image storage service, such as acloud service. Such images may be synced to other user devices of theuser.

The circular buffer 132 may store a sequence of images continuouslycaptured by the user device 102 at a frame rate. In such embodiments,the camera 126 of the user device 102 may continuously capture asequence of images without interaction from a user, and the imagescaptured by the user device may be stored in the circular buffer 132.Images may be retrieved from the circular buffer 132, such as by movingor copying the images from the circular buffer 132 into another memorylocation. Images may be transferred from the circular buffer 132, suchas by moving or copying the images to another location such as imagestorage 134 or an image storage device of image server 114. In someembodiments, a sequence of images may be stored in the circular buffer132 after the user device 102 enters a camera ready mode. For example,in some embodiments the camera ready mode may be entered by executing,on the user device 102, a camera application or an application that hascamera functionality. Accordingly, FIG. 1 depicts a portion 140 of thecircular buffer 132 illustrating captured images 142 stored in thebuffer 132. For example, the circular buffer portion 140 may store animage_(n) that may be captured at a point in time and stored, as animage_(n−1) captured one frame before image_(n), an image_(n+1) capturedone frame after image_(n) and so on. The frame rate of the camera 126 ofthe user device 102 may be affected by the light source 110. Forexample, the frame rate of captured images may be higher in relativelybright light and relatively lower in low light. In some embodiments, theframe rate may be 7 frames per second (fps) or less, 10 fps, or less, 15fps or less, 20 fps, or less, 25 fps or less, 30 fps or less, 35 fps orless, or any other suitable frame rate achievable by the camera of theuser device. In some embodiments, the user device 102 may include anambient light sensor or use image data generated by the camera module130 (e.g., as included in images stored in the circular buffer 132) todetermine the impact of light source 110 (e.g., to determine a lightingcondition). In such embodiments, the frame rate of captured images maybe associated with a determined lighting condition relative to apre-determined lighting threshold.

FIG. 2 is a schematic diagram depicting further details of the userdevice 102 and another portion 200 of the circular buffer 132 inaccordance with an embodiment of the disclosure. As shown by arrow 202,as time elapses the user device may continuously capture and storeimages in the circular buffer portion 200. The portion 200 illustratesvarious images 206 captured in corresponding locations 208 of circularbuffer portion 200 at a frame rate of a camera of the user device 102.For example, an image_(n) may be captured at a point in time (e.g., at ashutter button selection) and stored, followed by an image_(n+1)captured one frame later, an image_(n+2) captured two frames later, animage_(n+3) captured three frames later and so on. In some embodiments,the point in time at a shutter button selection may refer to thetimestamp closest to when a shutter button was determined to beselected. For example, if images are being captured at 30 fps, and ashutter button is selected between frames being captured, the oneclosest in time is considered to be the t=0 image. In other embodiments,the t=0 image could be the first image after the shutter button isselected or the image immediately preceding the selection of the shutterbutton selection. The images captured before a shutter button selectionmay be referred to as “pre-shutter” images). Similarly, images prior tothe capture of image_(n) may be captured and stored in the circularbuffer portion 200, such as image_(n−1), image_(n−2), image_(n−3), andso on. The images captured after a shutter button selection may bereferred to as “post-shutter” images. As will be appreciated, in someembodiments, the images 206 may include image metadata describing animage. For example, image metadata may include a resolution, a date andtime of image capture, a flash indication (whether the flash of thecamera was activated when the image was captured), a frame rate of thecamera when the image was captured, and the like.

As also mentioned above, the frame rate of the capture of images 206 maybe affected by the lighting conditions present when the images 206 arecaptured. For example, for relatively bright lighting conditions, theimages being captured and stored in the circular buffer portion 200 maybe captured at about 30 fps. In such an example, image_(n+1) may becaptured 1/30th of a second after image_(n), image_(n+2) may be captured1/30th of a second after image_(n+1), and so on. In another example, forrelatively low lighting conditions, the images being captured and storedin the circular buffer portion 200 may be captured at a lower framerate. For example, at a frame rate of 7 fps, image_(n+1) may be captured1/7th of a second after image_(n), image_(n+2) may be captured 1/7th ofa second after image_(n+1), and so on.

As explained below, embodiments of the present disclosure includeretrieval of multiple captured images from the circular buffer using aframe interval. The retrieved images may be presented to a user, and theuser can select a preferred image (also referred to as a “preferredimage” or “best shot”) from the retrieved images. FIG. 3 depictsretrieval of captured images from the circular buffer portion 200 usinga frame interval based on the frame rate of image capture. As mentionedabove and illustrated in FIG. 2, images 206 are continually captured ata frame rate and stored in corresponding locations 208 of the circularbuffer portion 202. FIG. 3 also depicts image storage locations 300 and302 of a memory of the user device 102 that may store images retrievedfrom the circular buffer portion 200.

At t=0 (indicated by 306) a user may take a photograph of the image 106by selecting the shutter button 116 of the user device 102, as describedabove. As described below, a t=0 image (“shutter” image) and alternativeimages captured before (a “pre-shutter” image) and after (a“post-shutter” image) the t=0 image (a “shutter” image) may be retrievedfrom the circular buffer 132. In some embodiments, the pre-shutter imageand post-shutter image may be retrieved based on either a timestampassociated with the t=0 image or the actual time the shutter button wasselected. In some embodiments, alternative images captured before andafter the t=0 image may be retrieved from the circular buffer portion200 using a frame interval based on the frame rate of image capture. Theframe interval may define the number of frames to “skip” in the circularbuffer when retrieving the alternative images. In some embodiments, theframe rate of image capture may be obtained from the camera (e.g., frommetadata provided by an image sensor). In some embodiments, the framerate may be an average frame rate, an instantaneous frame rate, oranother frame rate provided by the camera. In some embodiments, theobtained frame rate is a frame rate immediately prior to the selectionof the shutter button. In some embodiments, the frame interval may becalculated to provide a specific time interval between the t=0 image andthe alternative images. For example, in some embodiments the frameinterval may be calculated to provide a time interval of about 300 msbetween the t=0 image and the alternative image before the t=0 image andbetween the t=0 image and the alternative image after the t=0 image.

As shown in FIG. 3, alternative image 312 (a “pre-shutter” image)captured before the preferred image 308 may be retrieved from thecircular buffer based on a frame rate interval 312. In the embodimentdepicted in FIG. 3, the illustrated frame rate interval is five frames.Thus, as shown in FIG. 3, the first alternative image 312 (image_(n−5))is retrieved from a location 316 in the circular buffer portion 200 fiveframes before the capture of the t=0 image 308, such that four frames(corresponding to image_(n−1), image_(n−2), image_(n−3), andimage_(n−4)) are skipped. The first alternative image 312 may beretrieved from the circular buffer portion 200 and stored in thealternative image storage 302.

As also shown in FIG. 3, the image 308 in the circular buffer portion200 that corresponds to the time of t=0 may be retrieved and stored as adefault preferred image in preferred image storage 300. For example, asshown in FIG. 3, image_(n) in location 310 of the circular bufferportion 200 may be the image that temporally corresponds to the timet=0. The t=0 image 308 (a “shutter” image) is retrieved from thecircular buffer portion 200 and stored in the preferred image storage300. In some embodiments, the t=0 image 308 may be identified by apointer for the preferred image that points to a unique identifier forthe t=0 image. A second alternative image 314 (a “post-shutter” image)captured after the t=0 image 308 may also be retrieved from the circularbuffer portion 200. As shown in FIG. 3, the second alternative image 314(image_(n+5)) may be retrieved from a location 318 in the circularbuffer portion 200 that corresponds to a location 318 five frames afterthe capture of the t=0 image 308 such that four frames (corresponding toimage_(n+1), image_(n+2), image_(n+3), and image_(n+4)) are skipped. Thesecond alternative image 314 may be retrieved from the circular bufferand also stored in the alternative image storage 302.

In some embodiments, the retrieval and subsequent processing of the t=0image 308 may be prioritized over retrieval and subsequent processing ofthe alternative image 314. For example, the t=0 image 308 may beretrieved and a thumbnail of the t=0 image 308 may be generated andprovided to a user in a user interface of the user device afterretrieval and processing of the first alternative image 312 but beforethe retrieval and subsequent processing of the second alternative image314.

In some embodiments, the alternative image storage 302 may be adifferent location than the preferred image storage 300 described above.For example, in such embodiments the alternative image storage 302 maybe a “hidden” location inaccessible by the user, by certain applicationsexecuting on the user device, or both. In such embodiments, theretrieval and storage of the alternative images may be hidden from theuser. In some embodiments, the alternative images 312 and 314 may remainstored in the alternative image location until deleted by a user.

In some embodiments, each image retrieved from the circular buffer 132may be identified using a unique identifier associated with each image.Accordingly, the t=0 image 308 and the alternative images 312 and 314may each be associated with a unique identifier such that references toeach image may be performed using the identifier. In such embodiments, apointer to the preferred image may identify a unique identifier. Forexample, if the t=0 image is initially retrieved as the preferred imageand stored in the preferred image storage 300, a pointer to thepreferred image may identify the unique identifier associated with thet=0 image 308.

FIG. 4 depicts retrieval of alternative images from the circular bufferusing another frame rate interval based on the frame rate of imagecapture. As shown in FIG. 4, after retrieval of the t=0 image 308,alternative images 400 (a “pre-shutter” image) and 402 (a “post-shutter”image) respectively captured before and after the preferred image 308may be retrieved from the circular buffer based on a frame rate interval406.

In contrast to the embodiment illustrated in FIG. 3 and described above,the illustrated frame rate interval depicted in FIG. 4 is three frames.As mentioned above, in some embodiments, the frame interval may becalculated to provide a specific time interval between the t=0 image andthe alternative images. For example, in the embodiment depicted in FIG.4, a different frame rate of image capture may result in a shorter frameinterval to provide for a specific time interval (e.g., in someembodiments the same time interval specified for the embodiment depictedin FIG. 3). In this manner, a time interval between the t=0 frame andthe alternative frames may be maintained relatively constant with regardto changes in frame rate of a camera due to changing lightingconditions.

Thus, as shown in FIG. 4, the first alternative image 400 is retrievedfrom a location 408 in the circular buffer portion that is three framesbefore the capture of the t=0 image 308, such that two frames(corresponding to image_(n−1) and image_(n−2)) are skipped. As describedabove, the first alternative image 400 may be retrieved from thecircular buffer portion 200 and stored in the alternative image storage302 of the memory of the user device. Similarly, the second alternativeimage 402 may be retrieved from a circular buffer location 410 that isthree frames after the capture of the t=0 image 308, such that twoframes (corresponding to image_(n+1) and image_(n+2) are skipped. Thesecond alternative image 402 may also be stored in the alternative imagestorage 302.

In some embodiments, the frame interval described above and illustratedin FIGS. 3 and 4 may be symmetrical on either side of the t=0 image. Inother embodiments, the frame interval described above may beasymmetrical such that the frame interval before the t=0 image isdifferent from the frame interval after the t=0 image.

In some embodiments, an alternative image may be retrieved based on theabsence of a flash when the image was captured. For example, if the t=0image was captured with a flash (i.e., a flash of the user device wasactivated and illuminating the image), an image captured before the t=0image without a flash may be retrieved. FIG. 5 depicts retrieval ofalternative images from the circular buffer in accordance with anotherembodiment of the disclosure. As illustrated in FIG. 5, the image 106may be captured using a flash light source 500. As described above, whenthe shutter button is selected an image 502 (image_(f)) captured at timet=0 may be retrieved from a location 504 of the circular buffer andstored in a first image storage portion of the memory of the userdevice.

As also depicted in FIG. 5, alternative images captured before and afterthe t=0 image may be retrieved from the circular buffer. As shown inFIG. 5, the first alternative image 506 (image_(f−2)) and the secondalternative image 508 (image_(f+2)) captured before and after the t=0image 502 may also be retrieved from the circular buffer portion 200from respective buffer locations 510 and 512. In the embodiment depictedin FIG. 5, the alternative image 506 (a “pre-shutter” image) may beretrieved based on a duration 511 of time before the t=0 image 502 toensure that the alternative image 506 was not captured with a flash,i.e., the first alternative image 506 was captured before the flashactivated in response to the shutter button selection. For example, theimage 514 (image_(f−1)) that was captured using a flash (i.e., the flashwas illuminated for all captured images 502 and 514 may be skipped. Insome embodiments, the activation of a flash to capture an image may bedetermined from an indicated provided by or retrieved from a camera of auser device. For example, at or after the shutter button selection anindication of a flash may be obtained from the camera of the userdevice.

The alternative image 508 (a “post-shutter” image) after the t=0 image502 may be retrieved based on a frame interval 513 specifying a numberof frames before the t=0 image 502. The frame interval 513 may be basedon the image capture frame rate and may be determined as describedabove. For example, as shown in FIG. 5, the second alternative image 508is retrieved from a location 512 in the circular buffer portion that istwo frames before the capture of the t=0 image 502, such that one frame(corresponding to image_(f+1)) is skipped. As described above, thealternative images 506 and 508 are stored in an alternative imagestorage location 302 of the memory of the user device 102.

Moreover, although FIGS. 3, 4, and 5 are described with reference to onealternative image before the t=0 image and one alternative image afterthe t=0 image, in other embodiments more than two images may be selectedas alternative images. For example, in some embodiments, two images maybe retrieved before the 1=0 image, and two images may be retrieved afterthe t=0 images. In such embodiments, multiple frame intervals may beused and different numbers of frames may be skipped to determine thebuffer locations for the multiple alternative images.

FIGS. 6A, 6B, and 6C depict a process 600 for retrieving and presentingmultiple captured images in accordance with an embodiment of the presentdisclosure. FIGS. 6A and 6B also depict user interactions 602 with auser device. The process 600 may be implemented, for example, on aprocessor of a user device. The processor of a user device may performthe operations described below or, in some embodiments, may provideinstructions to perform one or more of the operations. Initially, a usermay initiate a camera ready mode of the user device (block 604), such asby executing of a camera application on the user device or anapplication on the user device that has camera functionality. Asdescribed above, images viewed by the camera are continually captured ata frame rate and stored in a circular buffer of a memory of the userdevice (block 606).

As shown in FIG. 6A, the frame rate of image capture in the circularbuffer is obtained (block 608)(this seems to have jumped from FIG. 6A to6B?). In some embodiments, a delta between the obtained frame rate and aprevious frame rate may be determined and compared to a threshold. Insuch embodiments, if the frame rate delta is greater than a threshold, aframe interval may be determined as described below in block 610(the #'sdon't seem to match the descriptions in FIGS. 6A AND 6B.)) If the framerate delta is less than the threshold, a previous frame interval may beused and a frame interval may not be determined using the obtained framerate.

As shown in FIG. 6A, based on the frame rate, a frame interval may bedetermined for retrieval of alternative images from the circular buffer(block 610). For example in some embodiments the frame interval may beselected such that a time interval between the alternative images andthe preferred image is about 300 ms. Next, if a user desires to take aphotograph the selection of the shutter button may be received (block614). The process 600 is further illustrated in FIG. 6B, as shown byconnection block A.

The activation of a flash when the shutter button was selected may alsobe determined (decision block 616), such as from an indicator providedby the camera. If a flash was activated, the first alternative imagebefore the t=0 image may be retrieved from the circular buffer based ona duration before the t=0 image. (block 618). If a flash was notactivated when the shutter button was selected, an image before thecapture of the t=0 image may be retrieved based on the determined frameinterval as the first alternative image (block 620). The retrieved firstalternative image may be stored in an alternative image location of amemory of the user device (block 622).

As described above, the image corresponding to the time of shutterbutton selection (t=0) may be retrieved from the circular buffer ofcaptured images (block 624). The t=0 image may be stored as thepreferred image (also referred to as “best shot”) in a preferred imagestorage (block 626). As shown by connection block B depicted in FIG. 6B,the process 600 and user actions 602 are further illustrated in FIG. 6C.

As shown in FIG. 6C, a thumbnail corresponding to the t=0 image may begenerated (block 628) and provided for display in a user interface(block 630). The user may view the thumbnail in the user interface as anindication of the photograph associated with the shutter buttonselection (block 632).

As described above, an image captured after the 1=0 image may beretrieved based on the determined frame interval as the secondalternative image (block 632). The retrieved second alternative imagemay be also stored in an alternative image location of a memory of theuser device (block 634).

Next, the t=0 image and the alternative images may be provided fordisplay to the user, such as in an image selection user interface (block636), to enable a user to select a preferred image (also referred to asa “best shot” image) for the photograph. For example, in someembodiments, the t=0 image and the alternative images may be displayedto the user in a user interface, and a user may view all three images(block 637). In some embodiments, the t=0 image may be selected as thedefault preferred image. After viewing the alternative images (block637), the user may decide to keep the default preferred image, the t=0image, as the preferred image for the photograph.

As shown in FIG. 6C, in some instances, the user may select analternative image as the preferred image (block 640). The selection ofthe alternative image may be received (block 638), and the alternativeimage may be stored as the preferred image (block 642). Additionally,the deselected t=0 image may then be stored as an alternative image(block 646) For example, in some embodiments, the user-selectedalternative image may be moved to the preferred image storage locationand the unselected 1=0 image may be moved to the alternative imagestorage location. In some embodiments, the alternative images may remainstored in the alternative image storage location until explicitlydeleted by the user (e.g., via the user interface in the mannerdescribed above). In some embodiments, the t=0 image and the alternativeimages may be named to reflect the selected preferred image. The firstalternative image (captured before the t=0 image) may be named using a“0.1” suffix and the second alternative image may be named using a “0.3”suffix. As the default preferred image, the t=0 image may be namedwithout a suffix. When an alternative image is selected as the preferredimage, the t=0 image may be renamed using a “0.2” suffix and theselected alternative image may be renamed without a suffix. In someembodiments, a pointer for the photograph may be modified to point to aunique identifier associated with the user-selected alternative image.

In some embodiments, the user-selected preferred image may betransmitted to a cloud storage service, (e.g., device networked storagedevice.) In some embodiments, other user devices of the user, such assmartphones, tablet computers, and so on, may synchronize with the cloudstorage service and may receive the user-selected preferred image forstoring in an image collection. Thus, other user devices may alsoindicate that the user-selected preferred image is the photographcaptured at t=0. In some embodiments, only one of the t=0 image, firstalternative image, and second alternative image may be transmitted toand stored by a networked storage device, such as based on one of imagesselected by the user as a preferred image. In other embodiments, the t=0image and the alternative images may all be transmitted and stored on anetwork storage device. In some embodiments, the preferred image and thealternative images may be compressed into a single image (e.g., a JPEGfile) for storing by the cloud storage service. In such embodiments, auser device may decompress the JPEG file to separate the preferred imageand the alternative images for viewing and editing. In some embodiments,user devices without the capability decompress the JPEG file may processthe JPEG file as a single preferred image without reference to the othercompressed data representing the alternative images.

FIGS. 7A-7E depict screens of a user interface 700 of a user deviceillustrating selection of a preferred image from a t=0 image andalternative images in accordance with the techniques described herein.FIG. 7 depicts a screen 702 of a user interface illustrating the initialdisplay of a t=0 image retrieved in response to a user selection of ashutter button of a user device. The user interface 700 may includevarious user-selectable elements, such as user-selectable buttons 704,such for switching to an image capture mode, sharing an image, editingan image. The user interface 700 may also include a user-selectablebutton 706 for entering an image selection mode that enables a user toselect a preferred photograph from among a t=0 image and alternativeimages. As shown in FIG. 7A, initially a thumbnail image 708 of the t=0image retrieved in response to the selection of the shutter button isdisplayed in the user interface 700. As described above, in someembodiments the t=0 may be retrieved and processed before retrieval andprocessing of the alternative images described herein to minimize delayin presenting the thumbnail image 708 to a user.

FIG. 7B depicts another screen 710 of the user interface 700illustrating an image selection mode in accordance with an embodiment ofthe present disclosure. As shown in FIG. 7B, the thumbnail 708 of thet=0 image may initially be displayed as the default preferred image. Aportion of the thumbnails 712 and 714 of alternative images may also bedisplayed on either side of the t=0 image thumbnail 708. For example,the alternative image thumbnail 712 displayed to the left of the t=0image thumbnail 708 may correspond to the alternative image capturedbefore the capture of the t=0 image, as determined according to thetechniques described above. Similarly, the alternative image thumbnail714 displayed to the right of the t=0 image thumbnail 708 may correspondto the alternative image captured after the capture of the t=0 image, asdetermined according to the techniques described above. The screen 710also depicts instructions 716 that may instruct a user how to move,select, or otherwise interact with the displayed images.

A selected image control, such as radio button 718, may be overlaid onthe images to indicate the image selected as the user's preferred image.Accordingly, for the currently selected t=0 image 708, the radio button718 may be selected to show that the t=0 image is selected as thedefault preferred image. The user interface may also include aconfirmation button 720 that confirms a selection of an image and, insome embodiments, exits the image selection mode. In some embodiments, auser may switch between the t=0 image 708 and the alternative images 712and 714 by performing a swipe gesture on the screen 710 in thedirections indicated by arrows 722 and 723. For example, a user mayswitch to the alternative image 712 by swiping to the left and switch tothe alternative image 714 by swiping to the right. In other embodiments,user interface controls or other gestures may be used to switch betweenthe images 708, 712, and 714. Additionally, in some embodiments a usermay delete an image by performing a swipe gesture in the directionindicated by arrow 724, (e.g., by “pushing” the image up until itdisappears from the screen 710). Similarly, in some embodiments a usermay save an image as a separate image by performing a swipe gesture inthe direction indicated by arrow 726 (e.g., by “pulling” the image downuntil it disappears from the screen 710). In some embodiments, theunselected images presented in the user interface 700 may remain storedon the user device until deleted via the user interface.

FIG. 7C depicts another screen 728 of the user interface 700illustrating a switch to the alternative image 714. For example, theuser may swipe in the direction indicated by arrow 722 to switch to thealternative image 714. As shown in FIG. 7C, a portion of the t=0 image708 is displayed on the left side of the screen 728 and the alternativeimage 714 is displayed in the center of the screen 728. In someembodiments, the switch between images may be visualized via ananimation responsive to a swipe gesture or other gesture or userinterface control that enables switching between images. As also shownin FIG. 7C, a radio button 730 may be overlaid on the alternative image714. The radio button 730 may be unselected if, as shown in FIG. 7C,another image (the t=0 image 708) is currently selected as the preferredimage.

FIG. 7D depicts another screen 732 illustrating selection of thealternative image 714 as the preferred image. As shown in 7D, the radiobutton 730 may be selected by a user to select the alternative image 714as the preferred image. A user may confirm a selection of a preferredphotograph by selecting the confirmation button 720. After confirmation,the user may be presented with the screen 734 depicted in FIG. 7E. Afterthe alternative image 714 is selected as the preferred image, the screen734 may be displayed to indicate that the alternative image 714 is thepreferred image for the photograph taken via the shutter buttonselection. As mentioned above, in some embodiments after selection of adifferent preferred image, the user-selected preferred image may betransmitted, via a network, to one or more servers of a cloud storageservice. In such embodiments, a user may access the t=0 photograph fromother user devices and view the user-selected preferred image as the t=0photograph.

The operations and processes described and shown above may be carriedout or performed in any suitable order as desired in variousimplementations. Additionally, in certain implementations, at least aportion of the operations may be carried out in parallel. Furthermore,in certain implementations, less than or more than the operationsdescribed may be performed.

Certain aspects of the disclosure are described above with reference toblock and flow diagrams of systems, methods, apparatus, and/or computerprogram products according to various implementations. It will beunderstood that one or more blocks of the block diagrams and flowdiagrams, and combinations of blocks in the block diagrams and the flowdiagrams, respectively, can be implemented by computer-executableprogram instructions. Likewise, some blocks of the block diagrams andflow diagrams may not necessarily need to be performed in the orderpresented, or may not necessarily need to be performed at all, accordingto some implementations.

These computer-executable program instructions may be loaded onto aspecial-purpose computer or other particular machine, a processor, orother programmable data processing apparatus to produce a particularmachine, such that the instructions that execute on the computer,processor, or other programmable data processing apparatus create meansfor implementing one or more functions specified in the flow diagramblock or blocks. These computer program instructions may also be storedin a computer-readable storage media or memory that can direct acomputer or other programmable data processing apparatus to function ina particular manner, such that the instructions stored in thecomputer-readable storage media produce an article of manufactureincluding instruction means that implement one or more functionsspecified in the flow diagram block or blocks. As an example, certainimplementations may provide for a computer program product, comprising acomputer-readable storage medium having a computer-readable program codeor program instructions implemented therein, said computer-readableprogram code adapted to be executed to implement one or more functionsspecified in the flow diagram block or blocks. The computer programinstructions may also be loaded onto a computer or other programmabledata processing apparatus to cause a series of operational elements orsteps to be performed on the computer or other programmable apparatus toproduce a computer-implemented process such that the instructions thatexecute on the computer or other programmable apparatus provide elementsor steps for implementing the functions specified in the flow diagramblock or blocks.

Accordingly, blocks of the block diagrams and flow diagrams supportcombinations of means for performing the specified functions,combinations of elements or steps for performing the specified functionsand program instruction means for performing the specified functions. Itwill also be understood that each block of the block diagrams and flowdiagrams, and combinations of blocks in the block diagrams and flowdiagrams, can be implemented by special-purpose, hardware-based computersystems that perform the specified functions, elements or steps, orcombinations of special-purpose hardware and computer instructions.

Conditional language, such as, among others, “can,” “could,” “might,” or“may,” unless specifically stated otherwise, or otherwise understoodwithin the context as used, is generally intended to convey that certainimplementations could include, while other implementations do notinclude, certain features, elements, and/or operations. Thus, suchconditional language is not generally intended to imply that features,elements, and/or operations are in any way required for one or moreimplementations or that one or more implementations necessarily includelogic for deciding, with or without user input or prompting, whetherthese features, elements, and/or operations are included or are to beperformed in any particular implementation.

Many modifications and other implementations of the disclosure set forthherein will be apparent having the benefit of the teachings presented inthe foregoing descriptions and the associated drawings. Therefore, it isto be understood that the disclosure is not to be limited to thespecific implementations disclosed and that modifications and otherimplementations are intended to be included within the scope of theappended claims. Although specific terms are employed herein, they areused in a generic and descriptive sense only and not for purposes oflimitation.

The invention claimed is:
 1. A method, comprising: receiving, at a firsttime, a user input that causes a mobile device to capture image data;storing, in a first memory device, a sequence of images represented bythe image data captured by the mobile device; determining a first imagefrom the sequence of images, wherein the first image was captured duringactivation of a light source of the mobile device; determining a secondimage from the sequence of images, wherein the second image is capturedbefore the first image before activation of the light source; andsending, to a second memory device, at least a portion of the image datarepresenting at least one of the first image and the second image. 2.The method of claim 1, wherein the second image is determined based atleast in part on a first time interval before the first image wascaptured.
 3. The method of claim 2, wherein the first time interval isbased at least in part on the frame rate.
 4. The method of claim 2,further comprising: determining a third image from the sequence ofimages, wherein the third image is captured after the first image wascaptured and during activation of the light source; wherein the thirdimage is determined using a second time interval after the first imagewas capture, the second time interval being different than the firsttime interval.